Hypotheses and Version Spaces

We consider the relation of a learning model described in terms of Formal Concept Analysis in [3] with a standard model of Machine Learning called version spaces. A version space consists of all possible functions compatible with training data. The overlap and distinctions of these two models are discussed. We give an algorithm how to generate a version space for which the classifiers are closed under conjunction. As an application we discuss an example from predicitive toxicology. The classifiers here are chemical compounds and their substructures. The example also indicates how the methodology can be applied to Conceptual Graphs. 1 Hypotheses and Implications Our paper deals with a standard model of Machine Learning called version spaces. But first, to make the paper self-contained, we introduce basic definitions of Formal Concept Analysis (FCA) [5]. We consider a set M of “structural attributes”, a set G of objects (or observations) and a relation I ⊆ G×M such that (g,m) ∈ I if and only if object g has the attribute m. Instead of (g,m) ∈ I we also write gIm. Such a triple K := (G,M, I) is called a formal context. Using the derivation operators, defined for A ⊆ G, B ⊆ M by A′ := {m ∈ M | gIm for all g ∈ A}, B′ := {g ∈ G | gIm for all m ∈ B}, we can define a formal concept (of the context K) to be a pair (A,B) satisfying A ⊆ G, B ⊆ M , A′ = B, and B′ = A. A is called the extent and B is called the intent of the concept (A,B). These concepts, ordered by (A1, B1) ≥ (A2, B2) ⇐⇒ A1 ⊇ A2 form a complete lattice, called the concept lattice of K := (G,M, I). Next, we use the FCA setting to describe a learning model from [2]. In addition to the structural attributes of M , we consider (as in [3]) a goal attribute w / ∈ M . This divides the set G of all objects into three subsets: The set G+ of those objects that are known to have the property w (these are the positive examples), the set G− of those objects of which it is known that they do not have w (the negative examples) and the set Gτ of undetermined examples, i.e., A. de Moor, W. Lex, and B. Ganter (Eds.): ICCS 2003, LNAI 2746, pp. 83–95, 2003. c © Springer-Verlag Berlin Heidelberg 2003 84 Bernhard Ganter and Sergei O. Kuznetsov of those objects, of which it is unknown if they have property w or not. This gives three subcontexts of K = (G,M, I): K+ := (G+,M, I+), K− := (G−,M, I−), and Kτ := (Gτ ,M, Iτ ), where for ε ∈ {+,−, τ} we have Iε := I ∩ (Gε ×M). Intents, as defined above, are attribute sets shared by some of the observed objects. In order to form hypotheses about structural causes of the goal attribute w, we are interested in sets of structural attributes that are common to some positive, but to no negative examples. Thus, a positive hypothesis for w is an intent of K+ that is not contained in the intent g− := {m ∈ M | (g,m) ∈ I−} of any negative example g ∈ G−. Negative hypotheses are defined accordingly. These hypotheses can be used as predictors for the undetermined examples. If the intent g′ := {m ∈ M | (g,m) ∈ I} of an object g ∈ Gτ contains a positive, but no negative hypothesis, then g is classified positively. Negative classifications are defined similarly. If g′ contains hypotheses of both kinds, or if g′ contains no hypothesis at all, then the classification is contradictory or undetermined, respectively. In [11] we argued that the consideration can be restricted to minimal (w.r.t. inclusion ⊆) hypotheses, positive as well as negative, since an object intent obviously contains a positive hypothesis if and only if it contains a minimal positive hypothesis, etc.